{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_position: 3\n",
    "title: Streaming\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Streaming With LangChain\n",
    "\n",
    "Streaming is critical in making applications based on LLMs feel responsive to end-users.\n",
    "\n",
    "Important LangChain primitives like LLMs, parsers, prompts, retrievers, and agents implement the LangChain Runnable Interface.\n",
    "\n",
    "This interface provides two general approaches to stream content:\n",
    "\n",
    "- `.stream()`: a default implementation of streaming that streams the final output from the chain.\n",
    "- `streamEvents()` and `streamLog()`: these provide a way to stream both intermediate steps and final output from the chain.\n",
    "\n",
    "Let’s take a look at both approaches!\n",
    "\n",
    "# Using Stream\n",
    "\n",
    "All `Runnable` objects implement a method called stream.\n",
    "\n",
    "These methods are designed to stream the final output in chunks, yielding each chunk as soon as it is available.\n",
    "\n",
    "Streaming is only possible if all steps in the program know how to process an **input stream**; i.e., process an input chunk one at a time, and yield a corresponding output chunk.\n",
    "\n",
    "The complexity of this processing can vary, from straightforward tasks like emitting tokens produced by an LLM, to more challenging ones like streaming parts of JSON results before the entire JSON is complete.\n",
    "\n",
    "The best place to start exploring streaming is with the single most important components in LLM apps – the models themselves!\n",
    "\n",
    "## LLMs and Chat Models\n",
    "\n",
    "Large language models can take several seconds to generate a complete response to a query. This is far slower than the **~200-300 ms** threshold at which an application feels responsive to an end user.\n",
    "\n",
    "The key strategy to make the application feel more responsive is to show intermediate progress; e.g., to stream the output from the model token by token."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Module: null prototype] { default: {} }"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import \"dotenv/config\";"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{=mdx}\n",
    "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
    "\n",
    "<ChatModelTabs />\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "|\n",
      "Hello|\n",
      "!|\n",
      " I|\n",
      "'m|\n",
      " an|\n",
      " AI|\n",
      " language|\n",
      " model|\n",
      " developed|\n",
      " by|\n",
      " Open|\n",
      "AI|\n",
      ".|\n",
      " I|\n",
      "'m|\n",
      " designed|\n",
      " to|\n",
      " assist|\n",
      " with|\n",
      " a|\n",
      " wide|\n",
      " range|\n",
      " of|\n",
      " tasks|\n",
      " and|\n",
      " topics|\n",
      ",|\n",
      " from|\n",
      " answering|\n",
      " questions|\n",
      " and|\n",
      " engaging|\n",
      " in|\n",
      " conversations|\n",
      ",|\n",
      " to|\n",
      " helping|\n",
      " with|\n",
      " writing|\n",
      " and|\n",
      " providing|\n",
      " information|\n",
      " on|\n",
      " various|\n",
      " subjects|\n",
      ".|\n",
      " I|\n",
      " don|\n",
      "'t|\n",
      " have|\n",
      " personal|\n",
      " experiences|\n",
      " or|\n",
      " emotions|\n",
      ",|\n",
      " as|\n",
      " I|\n",
      "'m|\n",
      " just|\n",
      " a|\n",
      " computer|\n",
      " program|\n",
      ",|\n",
      " but|\n",
      " I|\n",
      "'m|\n",
      " here|\n",
      " to|\n",
      " help|\n",
      " and|\n",
      " provide|\n",
      " information|\n",
      " to|\n",
      " the|\n",
      " best|\n",
      " of|\n",
      " my|\n",
      " abilities|\n",
      ".|\n",
      " Is|\n",
      " there|\n",
      " something|\n",
      " specific|\n",
      " you|\n",
      "'d|\n",
      " like|\n",
      " to|\n",
      " know|\n",
      " or|\n",
      " discuss|\n",
      "?|\n",
      "|\n"
     ]
    }
   ],
   "source": [
    "const stream = await model.stream(\"Hello! Tell me about yourself.\");\n",
    "const chunks = [];\n",
    "for await (const chunk of stream) {\n",
    "  chunks.push(chunk);\n",
    "  console.log(`${chunk.content}|`)\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's have a look at one of the raw chunks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessageChunk {\n",
       "  lc_serializable: \u001b[33mtrue\u001b[39m,\n",
       "  lc_kwargs: { content: \u001b[32m\"\"\u001b[39m, additional_kwargs: {} },\n",
       "  lc_namespace: [ \u001b[32m\"langchain_core\"\u001b[39m, \u001b[32m\"messages\"\u001b[39m ],\n",
       "  content: \u001b[32m\"\"\u001b[39m,\n",
       "  name: \u001b[90mundefined\u001b[39m,\n",
       "  additional_kwargs: {}\n",
       "}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chunks[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We got back something called an `AIMessageChunk`. This chunk represents a part of an `AIMessage`.\n",
    "\n",
    "Message chunks are additive by design – one can simply add them up using the `.concat()` method to get the state of the response so far!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessageChunk {\n",
       "  lc_serializable: \u001b[33mtrue\u001b[39m,\n",
       "  lc_kwargs: { content: \u001b[32m\"Hello! I'm\"\u001b[39m, additional_kwargs: {} },\n",
       "  lc_namespace: [ \u001b[32m\"langchain_core\"\u001b[39m, \u001b[32m\"messages\"\u001b[39m ],\n",
       "  content: \u001b[32m\"Hello! I'm\"\u001b[39m,\n",
       "  name: \u001b[90mundefined\u001b[39m,\n",
       "  additional_kwargs: {}\n",
       "}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let finalChunk = chunks[0];\n",
    "\n",
    "for (const chunk of chunks.slice(1, 5)) {\n",
    "  finalChunk = finalChunk.concat(chunk);\n",
    "}\n",
    "\n",
    "finalChunk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chains\n",
    "\n",
    "Virtually all LLM applications involve more steps than just a call to a language model.\n",
    "\n",
    "Let’s build a simple chain using `LangChain Expression Language` (`LCEL`) that combines a prompt, model and a parser and verify that streaming works.\n",
    "\n",
    "We will use `StringOutputParser` to parse the output from the model. This is a simple parser that extracts the content field from an `AIMessageChunk`, giving us the `token` returned by the model.\n",
    "\n",
    ":::{.callout-tip}\n",
    "LCEL is a declarative way to specify a “program” by chainining together different LangChain primitives. Chains created using LCEL benefit from an automatic implementation of stream, allowing streaming of the final output. In fact, chains created with LCEL implement the entire standard Runnable interface.\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "|\n",
      "Sure|\n",
      "!|\n",
      " Here|\n",
      "'s|\n",
      " a|\n",
      " par|\n",
      "rot|\n",
      "-themed|\n",
      " joke|\n",
      " for|\n",
      " you|\n",
      ":\n",
      "\n",
      "|\n",
      "Why|\n",
      " did|\n",
      " the|\n",
      " par|\n",
      "rot|\n",
      " bring|\n",
      " a|\n",
      " ladder|\n",
      " to|\n",
      " the|\n",
      " party|\n",
      "?\n",
      "\n",
      "|\n",
      "Because|\n",
      " it|\n",
      " wanted|\n",
      " to|\n",
      " be|\n",
      " a|\n",
      " high|\n",
      " f|\n",
      "lier|\n",
      "!|\n",
      "|\n"
     ]
    }
   ],
   "source": [
    "import { StringOutputParser } from \"@langchain/core/output_parsers\";\n",
    "import { ChatPromptTemplate } from \"@langchain/core/prompts\";\n",
    "\n",
    "const prompt = ChatPromptTemplate.fromTemplate(\"Tell me a joke about {topic}\");\n",
    "\n",
    "const parser = new StringOutputParser();\n",
    "\n",
    "const chain = prompt.pipe(model).pipe(parser);\n",
    "\n",
    "const stream = await chain.stream({\n",
    "  topic: \"parrot\",\n",
    "});\n",
    "\n",
    "for await (const chunk of stream) {\n",
    "  console.log(`${chunk}|`)\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{.callout-note}\n",
    "You do not have to use the `LangChain Expression Language` to use LangChain and can instead rely on a standard **imperative** programming approach by\n",
    "caling `invoke`, `batch` or `stream` on each component individually, assigning the results to variables and then using them downstream as you see fit.\n",
    "\n",
    "If that works for your needs, then that's fine by us 👌!\n",
    ":::\n",
    "\n",
    "### Working with Input Streams\n",
    "\n",
    "What if you wanted to stream JSON from the output as it was being generated?\n",
    "\n",
    "If you were to rely on `JSON.parse` to parse the partial json, the parsing would fail as the partial json wouldn't be valid json.\n",
    "\n",
    "You'd likely be at a complete loss of what to do and claim that it wasn't possible to stream JSON.\n",
    "\n",
    "Well, turns out there is a way to do it - the parser needs to operate on the **input stream**, and attempt to \"auto-complete\" the partial json into a valid state.\n",
    "\n",
    "Let's see such a parser in action to understand what this means."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{ countries: [] }\n",
      "{ countries: [ { name: \"\" } ] }\n",
      "{ countries: [ { name: \"France\" } ] }\n",
      "{ countries: [ { name: \"France\", population: \"\" } ] }\n",
      "{ countries: [ { name: \"France\", population: \"66\" } ] }\n",
      "{ countries: [ { name: \"France\", population: \"66,\" } ] }\n",
      "{ countries: [ { name: \"France\", population: \"66,960\" } ] }\n",
      "{ countries: [ { name: \"France\", population: \"66,960,\" } ] }\n",
      "{ countries: [ { name: \"France\", population: \"66,960,000\" } ] }\n",
      "{\n",
      "  countries: [ { name: \"France\", population: \"66,960,000\" }, { name: \"\" } ]\n",
      "}\n",
      "{\n",
      "  countries: [ { name: \"France\", population: \"66,960,000\" }, { name: \"Spain\" } ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,660\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,660,\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,660,000\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,660,000\" },\n",
      "    { name: \"\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,660,000\" },\n",
      "    { name: \"Japan\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,660,000\" },\n",
      "    { name: \"Japan\", population: \"\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,660,000\" },\n",
      "    { name: \"Japan\", population: \"126\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,660,000\" },\n",
      "    { name: \"Japan\", population: \"126,\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,660,000\" },\n",
      "    { name: \"Japan\", population: \"126,500\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,660,000\" },\n",
      "    { name: \"Japan\", population: \"126,500,\" }\n",
      "  ]\n",
      "}\n",
      "{\n",
      "  countries: [\n",
      "    { name: \"France\", population: \"66,960,000\" },\n",
      "    { name: \"Spain\", population: \"46,660,000\" },\n",
      "    { name: \"Japan\", population: \"126,500,000\" }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "import { JsonOutputParser } from \"@langchain/core/output_parsers\"\n",
    "\n",
    "const chain = model.pipe(new JsonOutputParser());\n",
    "const stream = await chain.stream(\n",
    "  `Output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key \"name\" and \"population\"`\n",
    ");\n",
    "\n",
    "for await (const chunk of stream) {\n",
    "  console.log(chunk);\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's **break** streaming. We'll use the previous example and append an extraction function at the end that extracts the country names from the finalized JSON. Since this new last step is just a function call with no defined streaming behavior, the streaming output from previous steps is aggregated, then passed as a single input to the function.\n",
    "\n",
    ":::{.callout-warning}\n",
    "Any steps in the chain that operate on **finalized inputs** rather than on **input streams** can break streaming functionality via `stream`.\n",
    ":::\n",
    "\n",
    ":::{.callout-tip}\n",
    "Later, we will discuss the `streamEvents` API which streams results from intermediate steps. This API will stream results from intermediate steps even if the chain contains steps that only operate on **finalized inputs**.\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\"France\",\"Spain\",\"Japan\"]\n"
     ]
    }
   ],
   "source": [
    "// A function that operates on finalized inputs\n",
    "// rather than on an input_stream\n",
    "\n",
    "// A function that does not operates on input streams and breaks streaming.\n",
    "const extractCountryNames = (inputs: Record<string, any>) => {\n",
    "  if (!Array.isArray(inputs.countries)) {\n",
    "    return \"\";\n",
    "  }\n",
    "  return JSON.stringify(inputs.countries.map((country) => country.name));\n",
    "}\n",
    "\n",
    "const chain = model.pipe(new JsonOutputParser()).pipe(extractCountryNames);\n",
    "\n",
    "const stream = await chain.stream(\n",
    "  `output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key \"name\" and \"population\"`\n",
    ");\n",
    "\n",
    "for await (const chunk of stream) {\n",
    "  console.log(chunk);\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Non-streaming components\n",
    "\n",
    "Like the above example, some built-in components like Retrievers do not offer any streaming. What happens if we try to `stream` them?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  [\n",
      "    Document {\n",
      "      pageContent: \"mitochondria is the powerhouse of the cell\",\n",
      "      metadata: {}\n",
      "    },\n",
      "    Document {\n",
      "      pageContent: \"buildings are made of brick\",\n",
      "      metadata: {}\n",
      "    }\n",
      "  ]\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "import { OpenAIEmbeddings } from \"@langchain/openai\";\n",
    "import { MemoryVectorStore } from \"langchain/vectorstores/memory\";\n",
    "import { ChatPromptTemplate } from \"@langchain/core/prompts\";\n",
    "\n",
    "const template = `Answer the question based only on the following context:\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "`;\n",
    "const prompt = ChatPromptTemplate.fromTemplate(template);\n",
    "\n",
    "const vectorstore = await MemoryVectorStore.fromTexts(\n",
    "  [\"mitochondria is the powerhouse of the cell\", \"buildings are made of brick\"],\n",
    "  [{}, {}],\n",
    "  new OpenAIEmbeddings(),\n",
    ");\n",
    "\n",
    "const retriever = vectorstore.asRetriever();\n",
    "\n",
    "const chunks = [];\n",
    "\n",
    "for await (const chunk of await retriever.stream(\"What is the powerhouse of the cell?\")) {\n",
    "  chunks.push(chunk);\n",
    "}\n",
    "\n",
    "console.log(chunks);\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stream just yielded the final result from that component. \n",
    "\n",
    "This is OK! Not all components have to implement streaming -- in some cases streaming is either unnecessary, difficult or just doesn't make sense.\n",
    "\n",
    ":::{.callout-tip}\n",
    "An LCEL chain constructed using some non-streaming components will still be able to stream in a lot of cases, with streaming of partial output starting after the last non-streaming step in the chain.\n",
    ":::\n",
    "\n",
    "Here's an example of this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import { RunnablePassthrough, RunnableSequence } from \"@langchain/core/runnables\";\n",
    "import type { Document } from \"@langchain/core/documents\";\n",
    "import { StringOutputParser } from \"@langchain/core/output_parsers\";\n",
    "\n",
    "const formatDocs = (docs: Document[]) => {\n",
    "  return docs.map((doc) => doc.pageContent).join(\"\\n-----\\n\")\n",
    "}\n",
    "\n",
    "const retrievalChain = RunnableSequence.from([\n",
    "  {\n",
    "    context: retriever.pipe(formatDocs),\n",
    "    question: new RunnablePassthrough()\n",
    "  },\n",
    "  prompt,\n",
    "  model,\n",
    "  new StringOutputParser(),\n",
    "]);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "|\n",
      "The|\n",
      " powerhouse|\n",
      " of|\n",
      " the|\n",
      " cell|\n",
      " is|\n",
      " the|\n",
      " mitochond|\n",
      "ria|\n",
      ".|\n",
      "|\n"
     ]
    }
   ],
   "source": [
    "const stream = await retrievalChain.stream(\"What is the powerhouse of the cell?\");\n",
    "\n",
    "for await (const chunk of stream) {\n",
    "  console.log(`${chunk}|`);\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we've seen how the `stream` method works, let's venture into the world of streaming events!\n",
    "\n",
    "## Using Stream Events\n",
    "\n",
    "Event Streaming is a **beta** API. This API may change a bit based on feedback.\n",
    "\n",
    ":::{.callout-note}\n",
    "Introduced in @langchain/core **0.1.27**.\n",
    ":::\n",
    "\n",
    "For the `streamEvents` method to work properly:\n",
    "\n",
    "* Any custom functions / runnables must propragate callbacks \n",
    "* Set proper parameters on models to force the LLM to stream tokens.\n",
    "* Let us know if anything doesn't work as expected!\n",
    "\n",
    "### Event Reference\n",
    "\n",
    "Below is a reference table that shows some events that might be emitted by the various Runnable objects.\n",
    "\n",
    ":::{.callout-note}\n",
    "When streaming is implemented properly, the inputs to a runnable will not be known until after the input stream has been entirely consumed. This means that `inputs` will often be included only for `end` events and rather than for `start` events.\n",
    ":::\n",
    "\n",
    "| event                | name             | chunk                           | input                                         | output                                          |\n",
    "|----------------------|------------------|---------------------------------|-----------------------------------------------|-------------------------------------------------|\n",
    "| on_llm_start         | [model name]     |                                 | {'input': 'hello'}                            |                                                 |\n",
    "| on_llm_stream        | [model name]     | 'Hello' `or` AIMessageChunk(content=\"hello\")  |                                               |                                   |\n",
    "| on_llm_end           | [model name]     |                                 | 'Hello human!'                                | {\"generations\": [...], \"llmOutput\": None, ...}  |\n",
    "| on_chain_start       | format_docs      |                                 |                                               |                                                 |\n",
    "| on_chain_stream      | format_docs      | \"hello world!, goodbye world!\"  |                                               |                                                 |\n",
    "| on_chain_end         | format_docs      |                                 | [Document(...)]                               | \"hello world!, goodbye world!\"                  |\n",
    "| on_tool_start        | some_tool        |                                 | {\"x\": 1, \"y\": \"2\"}                            |                                                 |\n",
    "| on_tool_stream       | some_tool        | {\"x\": 1, \"y\": \"2\"}              |                                               |                                                 |\n",
    "| on_tool_end          | some_tool        |                                 |                                               | {\"x\": 1, \"y\": \"2\"}                              |\n",
    "| on_retriever_start   | [retriever name] |                                 | {\"query\": \"hello\"}                            |                                                 |\n",
    "| on_retriever_chunk   | [retriever name] | {documents: [...]}              |                                               |                                                 |\n",
    "| on_retriever_end     | [retriever name] |                                 | {\"query\": \"hello\"}                            | {documents: [...]}                              |\n",
    "| on_prompt_start      | [template_name]  |                                 | {\"question\": \"hello\"}                         |                                                 |\n",
    "| on_prompt_end        | [template_name]  |                                 | {\"question\": \"hello\"}                         | ChatPromptValue(messages: [SystemMessage, ...]) |\n",
    "\n",
    "### Chat Model\n",
    "\n",
    "Let's start off by looking at the events produced by a chat model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[33m13\u001b[39m"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "const events = [];\n",
    "\n",
    "const eventStream = await model.streamEvents(\"hello\", { version: \"v1\" });\n",
    "\n",
    "for await (const event of eventStream) {\n",
    "  events.push(event);\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{.callout-note}\n",
    "\n",
    "Hey what's that funny version=\"v1\" parameter in the API?! 😾\n",
    "\n",
    "This is a **beta API**, and we're almost certainly going to make some changes to it.\n",
    "\n",
    "This version parameter will allow us to mimimize such breaking changes to your code. \n",
    "\n",
    "In short, we are annoying you now, so we don't have to annoy you later.\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at the few of the start event and a few of the end events."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\n",
       "  {\n",
       "    run_id: \u001b[32m\"ce08e556-e8e7-4bfb-b8c0-e51926fc9c0c\"\u001b[39m,\n",
       "    event: \u001b[32m\"on_llm_start\"\u001b[39m,\n",
       "    name: \u001b[32m\"ChatOpenAI\"\u001b[39m,\n",
       "    tags: [],\n",
       "    metadata: {},\n",
       "    data: { input: \u001b[32m\"hello\"\u001b[39m }\n",
       "  },\n",
       "  {\n",
       "    event: \u001b[32m\"on_llm_stream\"\u001b[39m,\n",
       "    run_id: \u001b[32m\"ce08e556-e8e7-4bfb-b8c0-e51926fc9c0c\"\u001b[39m,\n",
       "    tags: [],\n",
       "    metadata: {},\n",
       "    name: \u001b[32m\"ChatOpenAI\"\u001b[39m,\n",
       "    data: {\n",
       "      chunk: AIMessageChunk {\n",
       "        lc_serializable: \u001b[33mtrue\u001b[39m,\n",
       "        lc_kwargs: { content: \u001b[32m\"\"\u001b[39m, additional_kwargs: {} },\n",
       "        lc_namespace: [ \u001b[32m\"langchain_core\"\u001b[39m, \u001b[32m\"messages\"\u001b[39m ],\n",
       "        content: \u001b[32m\"\"\u001b[39m,\n",
       "        name: \u001b[90mundefined\u001b[39m,\n",
       "        additional_kwargs: {}\n",
       "      }\n",
       "    }\n",
       "  },\n",
       "  {\n",
       "    event: \u001b[32m\"on_llm_stream\"\u001b[39m,\n",
       "    run_id: \u001b[32m\"ce08e556-e8e7-4bfb-b8c0-e51926fc9c0c\"\u001b[39m,\n",
       "    tags: [],\n",
       "    metadata: {},\n",
       "    name: \u001b[32m\"ChatOpenAI\"\u001b[39m,\n",
       "    data: {\n",
       "      chunk: AIMessageChunk {\n",
       "        lc_serializable: \u001b[33mtrue\u001b[39m,\n",
       "        lc_kwargs: { content: \u001b[32m\"Hello\"\u001b[39m, additional_kwargs: {} },\n",
       "        lc_namespace: [ \u001b[32m\"langchain_core\"\u001b[39m, \u001b[32m\"messages\"\u001b[39m ],\n",
       "        content: \u001b[32m\"Hello\"\u001b[39m,\n",
       "        name: \u001b[90mundefined\u001b[39m,\n",
       "        additional_kwargs: {}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "events.slice(0, 3);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\n",
       "  {\n",
       "    event: \u001b[32m\"on_llm_stream\"\u001b[39m,\n",
       "    run_id: \u001b[32m\"ce08e556-e8e7-4bfb-b8c0-e51926fc9c0c\"\u001b[39m,\n",
       "    tags: [],\n",
       "    metadata: {},\n",
       "    name: \u001b[32m\"ChatOpenAI\"\u001b[39m,\n",
       "    data: {\n",
       "      chunk: AIMessageChunk {\n",
       "        lc_serializable: \u001b[33mtrue\u001b[39m,\n",
       "        lc_kwargs: { content: \u001b[32m\"\"\u001b[39m, additional_kwargs: {} },\n",
       "        lc_namespace: [ \u001b[32m\"langchain_core\"\u001b[39m, \u001b[32m\"messages\"\u001b[39m ],\n",
       "        content: \u001b[32m\"\"\u001b[39m,\n",
       "        name: \u001b[90mundefined\u001b[39m,\n",
       "        additional_kwargs: {}\n",
       "      }\n",
       "    }\n",
       "  },\n",
       "  {\n",
       "    event: \u001b[32m\"on_llm_end\"\u001b[39m,\n",
       "    name: \u001b[32m\"ChatOpenAI\"\u001b[39m,\n",
       "    run_id: \u001b[32m\"ce08e556-e8e7-4bfb-b8c0-e51926fc9c0c\"\u001b[39m,\n",
       "    tags: [],\n",
       "    metadata: {},\n",
       "    data: { output: { generations: [ \u001b[36m[Array]\u001b[39m ] } }\n",
       "  }\n",
       "]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "events.slice(-2);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chain\n",
    "\n",
    "Let's revisit the example chain that parsed streaming JSON to explore the streaming events API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[33m117\u001b[39m"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "const chain = model.pipe(new JsonOutputParser());\n",
    "const eventStream = await chain.streamEvents(\n",
    "  `Output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key \"name\" and \"population\"`,\n",
    "  { version: \"v1\" },\n",
    ");\n",
    "\n",
    "\n",
    "const events = [];\n",
    "for await (const event of eventStream) {\n",
    "  events.push(event);\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you examine at the first few events, you'll notice that there are **3** different start events rather than **2** start events.\n",
    "\n",
    "The three start events correspond to:\n",
    "\n",
    "1. The chain (model + parser)\n",
    "2. The model\n",
    "3. The parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\n",
       "  {\n",
       "    run_id: \u001b[32m\"c486d08d-b426-43c3-8fe0-a943db575133\"\u001b[39m,\n",
       "    event: \u001b[32m\"on_chain_start\"\u001b[39m,\n",
       "    name: \u001b[32m\"RunnableSequence\"\u001b[39m,\n",
       "    tags: [],\n",
       "    metadata: {},\n",
       "    data: {\n",
       "      input: \u001b[32m\"Output a list of the countries france, spain and japan and their populations in JSON format. Use a d\"\u001b[39m... 129 more characters\n",
       "    }\n",
       "  },\n",
       "  {\n",
       "    event: \u001b[32m\"on_llm_start\"\u001b[39m,\n",
       "    name: \u001b[32m\"ChatOpenAI\"\u001b[39m,\n",
       "    run_id: \u001b[32m\"220e2e35-06d1-4db7-87a4-9c35643eee13\"\u001b[39m,\n",
       "    tags: [ \u001b[32m\"seq:step:1\"\u001b[39m ],\n",
       "    metadata: {},\n",
       "    data: { input: { messages: [ \u001b[36m[Array]\u001b[39m ] } }\n",
       "  },\n",
       "  {\n",
       "    event: \u001b[32m\"on_parser_start\"\u001b[39m,\n",
       "    name: \u001b[32m\"JsonOutputParser\"\u001b[39m,\n",
       "    run_id: \u001b[32m\"34a7abe4-98ae-46ad-85ac-625e724468b1\"\u001b[39m,\n",
       "    tags: [ \u001b[32m\"seq:step:2\"\u001b[39m ],\n",
       "    metadata: {},\n",
       "    data: {}\n",
       "  }\n",
       "]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "events.slice(0, 3);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What do you think you'd see if you looked at the last 3 events? what about the middle?\n",
    "\n",
    "Let's use this API to take output the stream events from the model and the parser. We're ignoring start events, end events and events from the chain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Chat model chunk: \n",
      "Chat model chunk: {\"\n",
      "Chat model chunk: countries\n",
      "Chat model chunk: \":\n",
      "Parser chunk: {\"countries\":[]}\n",
      "Chat model chunk:  [\n",
      "\n",
      "Chat model chunk:    \n",
      "Chat model chunk:  {\"\n",
      "Chat model chunk: name\n",
      "Chat model chunk: \":\n",
      "Parser chunk: {\"countries\":[{\"name\":\"\"}]}\n",
      "Chat model chunk:  \"\n",
      "Parser chunk: {\"countries\":[{\"name\":\"fr\"}]}\n",
      "Chat model chunk: fr\n",
      "Parser chunk: {\"countries\":[{\"name\":\"france\"}]}\n",
      "Chat model chunk: ance\n",
      "Chat model chunk: \",\n",
      "Chat model chunk:  \"\n",
      "Chat model chunk: population\n",
      "Chat model chunk: \":\n",
      "Parser chunk: {\"countries\":[{\"name\":\"france\",\"population\":\"\"}]}\n",
      "Chat model chunk:  \"\n",
      "Parser chunk: {\"countries\":[{\"name\":\"france\",\"population\":\"67\"}]}\n"
     ]
    }
   ],
   "source": [
    "let eventCount = 0;\n",
    "\n",
    "const eventStream = await chain.streamEvents(\n",
    "  `Output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key \"name\" and \"population\"`,\n",
    "  { version: \"v1\" },\n",
    ");\n",
    "\n",
    "for await (const event of eventStream) {\n",
    "  // Truncate the output\n",
    "  if (eventCount > 30) {\n",
    "    continue;\n",
    "  }\n",
    "  const eventType = event.event;\n",
    "  if (eventType === \"on_llm_stream\") {\n",
    "    console.log(`Chat model chunk: ${event.data.chunk.message.content}`);\n",
    "  } else if (eventType === \"on_parser_stream\") {\n",
    "    console.log(`Parser chunk: ${JSON.stringify(event.data.chunk)}`);\n",
    "  }\n",
    "  eventCount += 1;\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because both the model and the parser support streaming, we see streaming events from both components in real time! Neat! 🦜\n",
    "\n",
    "### Filtering Events\n",
    "\n",
    "Because this API produces so many events, it is useful to be able to filter on events.\n",
    "\n",
    "You can filter by either component `name`, component `tags` or component `type`.\n",
    "\n",
    "#### By Name\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  event: \"on_parser_start\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"c889ec6f-6050-40c2-8fdb-c24ab88606c3\",\n",
      "  tags: [ \"seq:step:2\" ],\n",
      "  metadata: {},\n",
      "  data: {}\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_stream\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"c889ec6f-6050-40c2-8fdb-c24ab88606c3\",\n",
      "  tags: [ \"seq:step:2\" ],\n",
      "  metadata: {},\n",
      "  data: { chunk: {} }\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_stream\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"c889ec6f-6050-40c2-8fdb-c24ab88606c3\",\n",
      "  tags: [ \"seq:step:2\" ],\n",
      "  metadata: {},\n",
      "  data: { chunk: { countries: [] } }\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_stream\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"c889ec6f-6050-40c2-8fdb-c24ab88606c3\",\n",
      "  tags: [ \"seq:step:2\" ],\n",
      "  metadata: {},\n",
      "  data: { chunk: { countries: [ {} ] } }\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_stream\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"c889ec6f-6050-40c2-8fdb-c24ab88606c3\",\n",
      "  tags: [ \"seq:step:2\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: { countries: [ { name: \"\" } ] }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_stream\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"c889ec6f-6050-40c2-8fdb-c24ab88606c3\",\n",
      "  tags: [ \"seq:step:2\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: { countries: [ { name: \"France\" } ] }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_stream\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"c889ec6f-6050-40c2-8fdb-c24ab88606c3\",\n",
      "  tags: [ \"seq:step:2\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: { countries: [ { name: \"France\", population: \"\" } ] }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_stream\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"c889ec6f-6050-40c2-8fdb-c24ab88606c3\",\n",
      "  tags: [ \"seq:step:2\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: { countries: [ { name: \"France\", population: \"67\" } ] }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_stream\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"c889ec6f-6050-40c2-8fdb-c24ab88606c3\",\n",
      "  tags: [ \"seq:step:2\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: { countries: [ { name: \"France\", population: \"67,\" } ] }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_stream\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"c889ec6f-6050-40c2-8fdb-c24ab88606c3\",\n",
      "  tags: [ \"seq:step:2\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: { countries: [ { name: \"France\", population: \"67,081\" } ] }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_stream\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"c889ec6f-6050-40c2-8fdb-c24ab88606c3\",\n",
      "  tags: [ \"seq:step:2\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: { countries: [ { name: \"France\", population: \"67,081,\" } ] }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "const chain = model.withConfig({ runName: \"model\" })\n",
    "  .pipe(\n",
    "    new JsonOutputParser().withConfig({ runName: \"my_parser\" })\n",
    "  );\n",
    "\n",
    "\n",
    "const eventStream = await chain.streamEvents(\n",
    "  `Output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key \"name\" and \"population\"`,\n",
    "  { version: \"v1\" },\n",
    "  { includeNames: [\"my_parser\"] },\n",
    ");\n",
    "\n",
    "let eventCount = 0;\n",
    "\n",
    "for await (const event of eventStream) {\n",
    "  // Truncate the output\n",
    "  if (eventCount > 10) {\n",
    "    continue;\n",
    "  }\n",
    "  console.log(event);\n",
    "  eventCount += 1;\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### By type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  event: \"on_llm_start\",\n",
      "  name: \"model\",\n",
      "  run_id: \"0c525b62-0d00-461c-9d1e-1bd8b339e711\",\n",
      "  tags: [ \"seq:step:1\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    input: { messages: [ [ [HumanMessage] ] ] }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"model\",\n",
      "  run_id: \"0c525b62-0d00-461c-9d1e-1bd8b339e711\",\n",
      "  tags: [ \"seq:step:1\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \"\",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \"\", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \"\",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"model\",\n",
      "  run_id: \"0c525b62-0d00-461c-9d1e-1bd8b339e711\",\n",
      "  tags: [ \"seq:step:1\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \"{\\n\",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \"{\\n\", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \"{\\n\",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"model\",\n",
      "  run_id: \"0c525b62-0d00-461c-9d1e-1bd8b339e711\",\n",
      "  tags: [ \"seq:step:1\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \" \",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \" \", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \" \",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"model\",\n",
      "  run_id: \"0c525b62-0d00-461c-9d1e-1bd8b339e711\",\n",
      "  tags: [ \"seq:step:1\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: ' \"',\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: ' \"', additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: ' \"',\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"model\",\n",
      "  run_id: \"0c525b62-0d00-461c-9d1e-1bd8b339e711\",\n",
      "  tags: [ \"seq:step:1\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \"countries\",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \"countries\", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \"countries\",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"model\",\n",
      "  run_id: \"0c525b62-0d00-461c-9d1e-1bd8b339e711\",\n",
      "  tags: [ \"seq:step:1\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: '\":',\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: '\":', additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: '\":',\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"model\",\n",
      "  run_id: \"0c525b62-0d00-461c-9d1e-1bd8b339e711\",\n",
      "  tags: [ \"seq:step:1\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \" [\\n\",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \" [\\n\", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \" [\\n\",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"model\",\n",
      "  run_id: \"0c525b62-0d00-461c-9d1e-1bd8b339e711\",\n",
      "  tags: [ \"seq:step:1\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \"   \",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \"   \", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \"   \",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"model\",\n",
      "  run_id: \"0c525b62-0d00-461c-9d1e-1bd8b339e711\",\n",
      "  tags: [ \"seq:step:1\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \" {\\n\",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \" {\\n\", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \" {\\n\",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"model\",\n",
      "  run_id: \"0c525b62-0d00-461c-9d1e-1bd8b339e711\",\n",
      "  tags: [ \"seq:step:1\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \"     \",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \"     \", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \"     \",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "const chain = model.withConfig({ runName: \"model\" })\n",
    "  .pipe(\n",
    "    new JsonOutputParser().withConfig({ runName: \"my_parser\" })\n",
    "  );\n",
    "\n",
    "\n",
    "const eventStream = await chain.streamEvents(\n",
    "  `Output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key \"name\" and \"population\"`,\n",
    "  { version: \"v1\" },\n",
    "  { includeTypes: [\"llm\"] },\n",
    ");\n",
    "\n",
    "let eventCount = 0;\n",
    "\n",
    "for await (const event of eventStream) {\n",
    "  // Truncate the output\n",
    "  if (eventCount > 10) {\n",
    "    continue;\n",
    "  }\n",
    "  console.log(event);\n",
    "  eventCount += 1;\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### By Tags\n",
    "\n",
    ":::{.callout-caution}\n",
    "\n",
    "Tags are inherited by child components of a given runnable. \n",
    "\n",
    "If you're using tags to filter, make sure that this is what you want.\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  run_id: \"e7abe3de-2402-49f1-a9d7-622f6aa2f5b9\",\n",
      "  event: \"on_chain_start\",\n",
      "  name: \"RunnableSequence\",\n",
      "  tags: [ \"my_chain\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    input: \"Output a list of the countries france, spain and japan and their populations in JSON format. Use a d\"... 129 more characters\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_start\",\n",
      "  name: \"ChatOpenAI\",\n",
      "  run_id: \"4bc4598c-3bf9-44d2-9c30-f9c635875b31\",\n",
      "  tags: [ \"seq:step:1\", \"my_chain\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    input: { messages: [ [ [HumanMessage] ] ] }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_start\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"df3b3f2b-8b67-4eeb-9376-a21799475e8f\",\n",
      "  tags: [ \"seq:step:2\", \"my_chain\" ],\n",
      "  metadata: {},\n",
      "  data: {}\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"ChatOpenAI\",\n",
      "  run_id: \"4bc4598c-3bf9-44d2-9c30-f9c635875b31\",\n",
      "  tags: [ \"seq:step:1\", \"my_chain\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \"\",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \"\", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \"\",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_parser_stream\",\n",
      "  name: \"my_parser\",\n",
      "  run_id: \"df3b3f2b-8b67-4eeb-9376-a21799475e8f\",\n",
      "  tags: [ \"seq:step:2\", \"my_chain\" ],\n",
      "  metadata: {},\n",
      "  data: { chunk: {} }\n",
      "}\n",
      "{\n",
      "  event: \"on_chain_stream\",\n",
      "  run_id: \"e7abe3de-2402-49f1-a9d7-622f6aa2f5b9\",\n",
      "  tags: [ \"my_chain\" ],\n",
      "  metadata: {},\n",
      "  name: \"RunnableSequence\",\n",
      "  data: { chunk: {} }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"ChatOpenAI\",\n",
      "  run_id: \"4bc4598c-3bf9-44d2-9c30-f9c635875b31\",\n",
      "  tags: [ \"seq:step:1\", \"my_chain\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \"{\\n\",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \"{\\n\", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \"{\\n\",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"ChatOpenAI\",\n",
      "  run_id: \"4bc4598c-3bf9-44d2-9c30-f9c635875b31\",\n",
      "  tags: [ \"seq:step:1\", \"my_chain\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \" \",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \" \", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \" \",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"ChatOpenAI\",\n",
      "  run_id: \"4bc4598c-3bf9-44d2-9c30-f9c635875b31\",\n",
      "  tags: [ \"seq:step:1\", \"my_chain\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: ' \"',\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: ' \"', additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: ' \"',\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"ChatOpenAI\",\n",
      "  run_id: \"4bc4598c-3bf9-44d2-9c30-f9c635875b31\",\n",
      "  tags: [ \"seq:step:1\", \"my_chain\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: \"countries\",\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: \"countries\", additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: \"countries\",\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n",
      "{\n",
      "  event: \"on_llm_stream\",\n",
      "  name: \"ChatOpenAI\",\n",
      "  run_id: \"4bc4598c-3bf9-44d2-9c30-f9c635875b31\",\n",
      "  tags: [ \"seq:step:1\", \"my_chain\" ],\n",
      "  metadata: {},\n",
      "  data: {\n",
      "    chunk: ChatGenerationChunk {\n",
      "      text: '\":',\n",
      "      generationInfo: { prompt: 0, completion: 0 },\n",
      "      message: AIMessageChunk {\n",
      "        lc_serializable: true,\n",
      "        lc_kwargs: { content: '\":', additional_kwargs: {} },\n",
      "        lc_namespace: [ \"langchain_core\", \"messages\" ],\n",
      "        content: '\":',\n",
      "        name: undefined,\n",
      "        additional_kwargs: {}\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "const chain = model\n",
    "  .pipe(new JsonOutputParser().withConfig({ runName: \"my_parser\" }))\n",
    "  .withConfig({ tags: [\"my_chain\"] });\n",
    "\n",
    "\n",
    "const eventStream = await chain.streamEvents(\n",
    "  `Output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key \"name\" and \"population\"`,\n",
    "  { version: \"v1\" },\n",
    "  { includeTags: [\"my_chain\"] },\n",
    ");\n",
    "\n",
    "let eventCount = 0;\n",
    "\n",
    "for await (const event of eventStream) {\n",
    "  // Truncate the output\n",
    "  if (eventCount > 10) {\n",
    "    continue;\n",
    "  }\n",
    "  console.log(event);\n",
    "  eventCount += 1;\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Non-streaming components\n",
    "\n",
    "Remember how some components don't stream well because they don't operate on **input streams**?\n",
    "\n",
    "While such components can break streaming of the final output when using `stream`, `streamEvents` will still yield streaming events from intermediate steps that support streaming!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\"France\",\"Spain\",\"Japan\"]\n"
     ]
    }
   ],
   "source": [
    "// A function that operates on finalized inputs\n",
    "// rather than on an input_stream\n",
    "\n",
    "// A function that does not operates on input streams and breaks streaming.\n",
    "const extractCountryNames = (inputs: Record<string, any>) => {\n",
    "  if (!Array.isArray(inputs.countries)) {\n",
    "    return \"\";\n",
    "  }\n",
    "  return JSON.stringify(inputs.countries.map((country) => country.name));\n",
    "}\n",
    "\n",
    "const chain = model.pipe(new JsonOutputParser()).pipe(extractCountryNames);\n",
    "\n",
    "const stream = await chain.stream(\n",
    "  `output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key \"name\" and \"population\"`\n",
    ");\n",
    "\n",
    "for await (const chunk of stream) {\n",
    "  console.log(chunk);\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, the `stream` API doesn't work correctly because `extractCountryNames` doesn't operate on streams.\n",
    "\n",
    "Now, let's confirm that with `streamEvents` we're still seeing streaming output from the model and the parser."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Chat model chunk: \n",
      "Parser chunk: {}\n",
      "Chat model chunk: {\n",
      "\n",
      "Chat model chunk:  \n",
      "Chat model chunk:  \"\n",
      "Chat model chunk: countries\n",
      "Chat model chunk: \":\n",
      "Parser chunk: {\"countries\":[]}\n",
      "Chat model chunk:  [\n",
      "\n",
      "Chat model chunk:    \n",
      "Parser chunk: {\"countries\":[{}]}\n",
      "Chat model chunk:  {\n",
      "\n",
      "Chat model chunk:      \n",
      "Chat model chunk:  \"\n",
      "Chat model chunk: name\n",
      "Chat model chunk: \":\n",
      "Parser chunk: {\"countries\":[{\"name\":\"\"}]}\n",
      "Chat model chunk:  \"\n",
      "Parser chunk: {\"countries\":[{\"name\":\"France\"}]}\n",
      "Chat model chunk: France\n",
      "Chat model chunk: \",\n",
      "\n",
      "Chat model chunk:      \n",
      "Chat model chunk:  \"\n",
      "Chat model chunk: population\n",
      "Chat model chunk: \":\n",
      "Parser chunk: {\"countries\":[{\"name\":\"France\",\"population\":\"\"}]}\n",
      "Chat model chunk:  \"\n"
     ]
    }
   ],
   "source": [
    "const eventStream = await chain.streamEvents(\n",
    "  `output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key \"name\" and \"population\"`,\n",
    "  { version: \"v1\" },\n",
    ");\n",
    "\n",
    "let eventCount = 0;\n",
    "\n",
    "for await (const event of eventStream) {\n",
    "  // Truncate the output\n",
    "  if (eventCount > 30) {\n",
    "    continue;\n",
    "  }\n",
    "  const eventType = event.event;\n",
    "  if (eventType === \"on_llm_stream\") {\n",
    "    console.log(`Chat model chunk: ${event.data.chunk.message.content}`);\n",
    "  } else if (eventType === \"on_parser_stream\") {\n",
    "    console.log(`Parser chunk: ${JSON.stringify(event.data.chunk)}`);\n",
    "  }\n",
    "  eventCount += 1;\n",
    "}"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Deno",
   "language": "typescript",
   "name": "deno"
  },
  "language_info": {
   "file_extension": ".ts",
   "mimetype": "text/x.typescript",
   "name": "typescript",
   "nb_converter": "script",
   "pygments_lexer": "typescript",
   "version": "5.3.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
